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ABSTRACT 
The goale of the computation that extracts disparity fro* pairfl 
of pictures of a scene are defined, and the conetralnte lupoead upon that 
computation by the *nre*- dimensional structure of the uorPd are 
determined. F*pre5-jing [he computation as a grey-Sevel c&fr 1 1 a t i on 9* 
S;hown to be Inadequate. A precise expression of the goale of the 
computation is possible in a I on- level synodic visual processor] the 
constraints trans I at* In this environment ta p'e-requ isi tea an the 
binding of disparity values to Iqh- level synbols. The outline of a 
net hod based on this is given. 
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Introauctic^ 



Commercial pressure* have lad to con? I Oerabl e interest In the 
automatic ex t r 3c 1 1 on at disparity information from pairs of picture* of a 
acene. Since 13G8, there has been available a nachlna, the Hi I d-Haytheon 
B& stereoma* automated plotter, which can drau a contour *ap fro« tuo 
atrial photography The machine corbel a tea intensity measurements 
Obtained over local scans wiane on the two l^ageg: tha scan p-atha are the 
machine's current appro* inat ion to tha contour 1 1 net [i.e. lines of 
constant disparity), and frit failures or correspondence between tha scan* 
on tha two imaoes are used to improve tha appr a* i mpt Jen+ Adequate 
accuracy, if achieved at alfl, is raached uithin shout ai* iterations. 

flachines that Ms* to assign disparity values to an image by 
performing cornel at Ions between intensity arrays are subject to 
troublesome problems due to local minima! nV 1 , Kidnde & Assda 11973) 
describe the pron-lens. and have recently ! mil C t7 C r ted soire nays of 
avoiding (hem,. Iheir principal cures are 111 to Correlate the two fmagsa 
using local averages tafcen over regions That Ore initially relatively 
large, and uhich are aubseojJent ly reduced in *(zo as the solution I* 
approached; and (ii> to avoid ln C jl ulnlma trap* by introducing a sua I p 
amount of gaussian noise into the litag«, Theae techniques reduce 
considerably the Incidence of false assignments, tout they fall to remove 
them a l together* 

There has been less progress in the study of parallel a Igor 1th* * 
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for Making use of disparity inF^rmat ion, despite cone-'der-abl e r«cent 
intei-oat in the protegeing of disparity informat ion by the visual pjy«tem f 
(Barton, B I akemore * Petti grew (1367], Julasz 113713); Julasz 11371 
paS4l h and Sperling 113701 have bath suggaated poaaihlc schenas. Julesz'i 
model is informal in nature^ being hore phenonencildgika I then 
computational: it is very interesting because despite rts groat 
ainpl icl t^ F it displays an astonishing nunbcr of the proper t i ea that are 
exhibited fry the human disparity processing tyatem. Sperling's Modal is 
more formal + but it ia difficult to tall hnu uall it nould mdtIc. Thla la 
a prcblee, with alt complex parallel fcethods; they are vary exptntive to 
simulate, and it is extremely di fficul t to dar i ve ana I yt I ea I iy. fro** 
system urth cortple* non- linear co»por,enti, quantities that tnu I d ba 
measured expar i merral ly. About the besl one can hope tc do at present ig 
to atatc criteria that distinguish one tm\ly of nethoda tro» another h 
and atk whether those criteria are satisfied by the particular nathcd 
that u* u-se, 

Thi* note enquires about the e*act nature of the diaparlty 
computation. Jt io In sane sense a correlation and becausa that ig 
common knowledge and fairly precise fc a deeper character Seat j or haa not 
been aou-ght. Bui In order to formulate a method of carrying it out, one 
needs to be very precise about the goals of the conputat ign, and about 
the constraints imposed upon it by the structure of the three-dimensional 
uorld. Unless one use* a net hod that is baund pn aM and only tft* correct 
assumption*, there will be equations in uhfcr, it will far I 
urmecessar i ly. 
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Mea surin g dja^rliy. 

If a scene Is photographed tram tug alightly fllffsrtnt position*, 
the rela(ive positions of the objects in the scene will differ slightly 
on the tuo images. The d Iserepaneii* of Intelsat arise from the different 
distances of the Objects fro* the viewing position* and measurements <jt 
the discrepancies contain Uiefui information about the relative distances 
of the objects. The term binoeutar disparity refers to the difference in 
the angle- from each eye to a paint in the scene, imjauursd relative to 
some suitably cb-ooen angle of convergence. The central difficulty in 
defining uhat is meant by the process of extract lug binocular disparity 
tr-ofc an ipage is that disparity has to r*f*r to a physical entity - a 
po I n I »n a visible surface - yet it appears that ue cohpute it St a lever 
far befou that St uftich ths world is described in terms at Surfaces and 
Object a Uulesz 1971]. ]t Lias probably this fact that made so surprising 
Julesz'e conclusion that disparity assignment is a I ou- I eve I computation. 

In order to compute disparity correctly, the fpllouing step* ttiiat 
be carried outi firstly, a particular location on a surface in the Scene 
Must be located In qn« image; secondly,, the identical lotatipri *wwt be 
identified in the Other inapt; thirdly, ths relative position* of the 
tuo images of that location Must be measured. The most interesting! and 
nost troublesome part of the process concerns the sal set ion of a location 
on the vieued surface, and the identification of It* tMO images. The 
difficulty is that the choice of a point on the surface must be nade fro* 
the lmagejt if It could be chosen in soms absolute way - by lighting It 
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up at that point for e*ample - the problem would fte simple, 

Ue are now in a position to Under a tand uhy the disparity 
computation is not th* aame thing as a grey-level cor re I at ion. The reason 
is that grey- I eve I fteasurenents correspond to properties of the inane , 
rather than to properties of the Qfciect s being vieued. An (x, y| co- 
ordinate pair on art image is an artefact of the transducer, aince it does 
riot define a point on a physical surface in a way that a I long it to be 
identified on the ether i*ags. The mast glaring avatiple of th* failure 
ia the case uhe.re an imagra point corresponds to two surface points, the 
neBr*r Of unich is transparent or translucent! a goldfish in a BOnd ia 
one such case, where the uater surface and the gold Hah are 
simultaneously visible, Qthsr swamp lea are presided by figures 5i7*l and 
6.3,2 Of Julesz {13711. But the argument applies equally wall to the 
case of a aingle visible but face, and its consequence is that grey- level 
.■atehing methods ar& incorrect. On simple imagn^ the method ui I | 
succeed, oecause it re dose anough to the Fight i dear but as HbH #t al. 
(19731 have found, it ui 1 1 not succeed on coiplex inagea because it is 
based on incorrect premises. Their technique of introducing local 
smear i ny may be viewed as s us y of beginning to Identify a point in the 
image u i th a point on the physical surface [by adding additional 
constraints on uhat is matchedh in bo far aa it does SO, (he method ui I I 
become more reliable, out it If probably better to alt^ck the underlying 
issue d i rect I u> 
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The u» g_± |c*4 -level aumbo I a 

In order to formulate the disparity ccfiputat ion in a usable HS-y, 
uifl therefore need to be able to rdantify surface points from the tuo 
images > audi match them up. It is d&arly fruit I ess to try to latin I point s 
of 3 amaalft feati>relaBS surface* but if that turf ace contains a scratch, 
boundary* or other identifying physical mark Uhich produces a 1 KB I SfKf 
fairly sharp change in re-H tctance, that change in reflectance may be 
used to define the surface pointn Provided that th? chgngg ifi reflectance 
hat been identified! and dSScrifc-ed separately in the tl« images, the 
resulting descriptions Mill corraapond to an underlying physical reality* 
The computation of such a low- 1 eve I descr I pt Jen from one ifiaga has been 
dealt ulth at length elsevihare (harr 1974a & 1374b) , *ixf in called ths 
Ion- I eve I syjmoolic represenlat i on of an fmaga. Hence Me see that provided 
Stereo matching takes place betueen two lou-ltvel symbolic description, 
it 15 a ufl I | -founded operation* 

Finally* ue need to ask Whether any reasonably complex 
■easurem«nt ecu Id he used - or i g there something special about a Ion- 
level symbolic descr i pt i en? Simple Cell-like measurements are for example 
nearly auitabU, because they are SC-r*&tilws quite near tq igu- level 
assertions: but Mhen aesigning ditparlty values to simple eel Is, ere 
■toots a I I the usual problems aasociated ul|h measurements - a uhole est 
Of Simple eel la. a( neighbouring positions and! or iantat i-ona, corresponds 
to the untfsr lying Scratch eh* MhatiVtr In the nage, and i ► ig that 
comples* which neada to be matched against the corresponding coup lew 
derrvod from the ether image. El the important matching step 1g carried 
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Figure K L denote: the collection of left-image symbols, and R the collection of 
right-image symbols, These are connected through the set o-f disparity value 
symbol 5. The sets DL And Dr referred to in the text are show in the figure 



symbol It disparity computation B 



out on each Individual sinple eel I Tin.vijruHsnt _ the computation pecome* 
very uneconomica I ► Hence -one may expect that uh*n disparity ia actually 
assigned, the procEss opera*** on a very Iow-IeveI symbolic description* 
This method ui M fail only when |h Q lou-levet decryption* obtained fro* 
the two imagsa are very different! but thin is comparat iva ly rar*. and 
one seems to ncticE It when It happens. In any casa, this cireuwatante 
cannot be rio-aft with at the same very hj*j level* 

The problEm has nou been reduced to the co*iparasan at tuo lo*J- 
level symbolic descriptions, and ths aaaignrrent of disparity values to 
pairg of symbol*, drawn appropriately front each image. He turn now to 
•stamina briefly the ru^g and conatrainta to which thia process ia 
subject. 
The "ufic once" Cond ition 

Ue hawa Been tnat an element of a I on- lev* I synpnMc daacription 
of an image corresfiondn to a physically identifiable entity: in a nay that 
an image CO-ordinate does not, and In which neaaurepents made on 9rt image 
only apprpwihate. Th'«s allows one to atata tha firat condition tnat 
controls the matching of tuo lou-luvcl symbol ic deacr i pt i ons* It ra that 
each ioH-fevel synhpl should be aaaignedi exactly ©no diaparlty value, 
which in turn 1 up I i es that it should oe associated with at moat one 
eymbol computed from tha other rmaga. Thia ia called the "uae-ooce" 
condition, and it ia non- linear, 

The use-qnoe condition may be implemented ic the following way* 
Let L be the set of all left-irage law-level aymhglo, and ret R be the 
set of all right-ifcage lau-le*e| symbols. To each element * in L* there 
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correspond several element* | n fl, orH for 4K h of the poH-aible disparity 
values; and for each e-Jsmiirtt y i n fi, there is a corresponding set of 
elements in L, This situation i s. illustrated in figure. 1, The hatching 
of one element from L with on* from R corresponds to the aeBignment of a 
Single disparity value- to both elements, ao I at us introduce a third set 
that eons i a 1 3 of col Tec lions of etehenta representing all of the 
possible disparity values that nau, be bound to Each lon-lovel «y#bo I , In 
principle, one needa one such collection for each low-leva I eymbol. 
thmjgh members gf L and R ahara el events in D in the appropriate Nag {see 
beloul. The ua«-qm;g condition trenalatcs into the constraint thai each 
element of I, and of R *ay be bound to at lost one element of □. 

In practise, the set D will be very large unleee eteps are token 
to sennomise nn the number o-f ynjtfc that are necessary to- represent the 
disparity values; ao let us consider how dieparity representing units may 
ft* shared between several elamanta of L [sayl, □ has to be large enough 
ao that (at each Inu-lavel symdol can find an unused collection in □ that 
can be ueed far representing i ta disparity: and fuJ th* correspondence 
toe-tween L and R throuojh U Is MtH-def fned. To accomplish thin. thH 
collection of teft- lor right-) irage Symbols that ahere a given 
d i apar i ty-rr!f;r^5cnt i ng unit should have the property that it is very rare 
for two to be (lrpvoKed ai nu I taneously by (he image. 

Uhen one considera hou to construct a parallel network that 
implements thr, use-once condition, it Is af:ii.ir«nt that at least three 
variables must be acconodated: aieension and declination in the visual 
'■eld. and disna-n.). f, adhsfdctory a'ranoflrrfln 1 : 'ft nhriur ir figjre 2i In 
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Figure 2. One poss-iSle geometrical arrangemEnl of the get D of symbpfa 
fpr disparity values is to arrange Q in stripes ot Constant disparity. 
The figure ehous hou this may he done, The stripes running nearly 
horizontally contain units uith constant disparity dl< The x-caardinata 
in the image is ripreaented by the H-coordinsta In the figure, and the y- 
Coordinate in the image e-srresno'ioo roughly to the y-CPordinate in the 
figure. The y-a*is Is magnified in the figure because a given y 
Coordinate in the image needs ta Lie reur-jjgerttad a t each disparity. The 
point of the figure 15 to show how lou-level left- and right-inage 
auhbolB share elements of Q. Ths seta Dl arid Or, that are described in 
the tftut and shown in figure 1, are arranged to crosi ona another, and 
divide □ up in tuo different ways, Ths connexions that i»ple.lnBnt the usa- 
once condition run along these tMO directions. The connexion* that 
implement ths suggestion Interaction run mora nearly para I te I to the 
stripes, and they join stripes Cor respond i ng to the save disparity (di> 
in neijhb-ouring positions in ths ^isuar field. The conne* I one that 
*easur# local goodneaa-of-f i t Mould have an undirected d i atr iPut ion. 
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this, the units representing disparity values- arq arranged in stripes of 
constant disparity. The collections in Q that represent disparity values 
for Itft-rMago symbols (LI I ■« along ths diggon&l lima mgrkad 01; and 
thpetf *c"" right-i&iaga symbols He along the opposing set of I rn*e 0r + 
Thys j 9 ^ivirtttf up In tuo ways into disparity representing units,, **icK 
are nl%ui taneousiy shared in an appropriate fashion by L and R. The 
connexions that implement the use-once condl rlon are e I ear ly jarkedi they 
ryn along Dt and Dr, joining places that ee-nta-n rcprosmt&tTont of all 
possible disparity values that could be bound to a glver> left- or right- 
generated syrfticl. (In a neural inplenentat ion of this 3chene> aucta 
connexions would be Inhlblt0ry»l 

There are interesting d 1 H erences between the I up I ioat iona tor 
neurophysiology of these ideas 4 and at the model gf Julesi. Firstly, the 
important part at the compu Ta t I orv involves cons trgints on the disparity 
values that may be hound to Ion-level symbol^ the magnets In Julesi'e 
node I aeem however to correspond to rather gnspecific local disparity 
values, and we saw earlier that the disparity compytatiprt tannot be 
accurately expressed in these terms. The second point reste on the aau 
in which di spar ity-repre?en ting sets In D are assigned to L and to R+ Tho 
■ost economical M3y of forming the col I act inns nf low-level symbols* that 
are to uae the aame disparity-representing units. 1 4 probably to group 
together all synhcls that describe a small region and a small orientation 
range* QVily very rarely will two such aynbola Pe used simultaneously. If 
some achetie of this sort were being used, it would account for the 
exiatence of eel I a that behave aensitively Co disparity, but are 
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relatively tolerant to (job] tin and orientation (Hubst S Ur^l 1978k 
Furthermore, If mwlj be uithin these mnita that the main disparity 
computation ie being carried ™t. arni between which the governing 
connexions should U* tasde, dnn wu td not enpecl lo find ather eel l, T 
express ing a free-floating disparity value in a 'region* of the Inge, 
Because the Usenet of the computation require* that it be carried Out on 
bindings to lau-l ave | aynfcolfi. The model of Jy| flBZ uould, I thin*, (cad 
one to GHpflct such calls. 

Dj_aparj_tL| 13 contlm iqua almos t everywhere 

The use-once condition must be iatisf led bg the f I na I assignment 
nf disp.ir. t y va1ii# E to the Io^IovbI oymbollc descr i pt I one, Pitt It is not 
(such help in finding It* When applied to a randan dot stereogram, it 
Mill ensure that the description of each dot, or groyp of dote, in one 
Irtge is mated with not ecro than one slellar description computed from 
the other image; and a solution that satisfies this and leaves very feu 
dDte cut Mill firobaulLj OB correct tees the ne*t condition}. But there ie 
another ueeful properly of the rsai *r I d that can be introduced with 
advantage to speed the analysis. U ie that ixcept at object boundaries 
disparity ie a fund I on that varig 5 snoothly ever the inane. Fine 
teHture h nhich is the best source of disparity information about a 
surface, will be represented St the I ones t level by assertions about vsrg 
email features; and sweep t at object boundaries, the disparity value* 
that become bowid to neighbouring nymbols ui|l p e about the same. This 
fact allows the extent* of an interaction that "propoftSs" the disparity 



symbol ic di soar! fry computation 1* 



used at one point a a S strong candidate for the value- 31 neighbouring 
points: it corresponds in Julaaz 1 * C1971} Model to the lateral coupling 
provided by lh« 5113 1 I springs that join adjacent magnets, The 
iBfjIementafiori of thiy constraint in Sperling's (1979J fiodo I is obacure. 

Tbn- f tip I edentat 1 an af a suggestion Is «ne of those questions that 
it is not profitable to pur bus in detail, because Of the difficulty in 
testing, * r thcr physio I ogi ca I Py or computational I Uji the cone I us lor a to 
which one nay be led* I Shal I therefore make only three points stiout It. 
The first is thai, in principle, one uould like a suggestion to influence 
th« route to a solution, without disturbing the values In the solution 
□nee they pre found* This impliaa a time -dependence in ths interaction. 
Secondly, in order that the solution l»ay be stable, one uquIc! probably 
ah go need to add a seal) DC component , The third point, antf erne Hhich 
may actually be useful. Concerna tha geometry of the auggesHon 
interactions, and this is ahown in figure 2. There are tonne k j o<n p between 
all dieparity units that represent a i hilar diaparity values^ and that 
refer to symbols in neai-hy portions of tha visual field. In a neurei 
implementat ion. they uoul<J be excitatory. 

Goodie as -o£- f It 
The final important aspect of disparity r»eaeureiten t It the 
ojueation of hou sat iaf actor y a solution is* JuleSX eUphaeieed thE need 
for auch a measure, and in hi a modal, it corresponds to the total 
potential energy in the tuo, super imposed aseenblies of magna t a* Sperling 
C197B!l also used a potential energy neaiura in his foreulation, Juletz 
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ahoucd that in an ambiguous stereogram, ne psreeiva the better-correlated 
solution even If both have quit* high correlation This ia good evidence 
that tne matching proeeaa is parallel rather than serial f and that the 
goodnees-of-f i t measurs ia computed on a local caeie. 

In an I mp I g«, er > ta t i on of the kind that »e ara rti scussl rip, the 
gooffneee-of-f it of a aclution nay he measured by the proportion of feft- 
and right- image gymbo I b that become bound to disparity values- jn a 
perfect solution, the proportion mil b ft l H t and an inappropriate 
disparity binding u\ M have the effect of depressing the proper t ion of 
correct bindings in i tt neighbourhood. The local goodness-Of-f 1 1 function 
would affect the confidence with which disparity aasigmients are hade 
locally - i rB - r the strength uith which lhr:y are asserted - uh | ch in turn 
would affect the potency uith which they are suggested to nearbu regions. 
The goodness-ot-f i t function uo^ld tharaforE be Iflplemented by a unit 
that depress&d the local d lapar i ty-represent tag unite bg an amount that 
depended upon (he proportion of image synbols in the vicinity that have 
besn assigned disparity vgluen, I do not see haw to test for the preeence 
Of such units, except bg trial -9nd-error. 

Summar u ej the dispa'. [tu intera ct iona 

Tho interactions described above are r»n drawn together, and the 
conditions that are necessary to an i up lenient at ion of ttiia kind are made 
e*pl iel t. 

U) The disparity assignment ii made ie a result of a nstcblnfl operation 
performed on two )ou-le*el symbol i c doscript I one< computed I ndepandent 1 g 
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from the l»ft and right images. 

12J The match-ino. is i dp lamented toy applying conditions to the process of 
binding symbolic disparity descriptors to tha tou-Eevel symbols* Those 
constraints are trie use-once condition, the suggestion interaction, ami 
■ax ■ m< sing the goodness of fit. 

53 J Th e uaa-cnce nonaction requirEB interacllona nhoae geometry appears 
In figure 2. These interactions inhibit the confidents of those 
as*rgn«ranta that they connect. 

te) The suggest ion interactions have (he geDfcatry shoun also In figure 2. 
They connect disparity descriptors that represent similar disparity 
valves, and that are capable of being Pounci lo I on- level symbols 
referring to neighbouring positione in the visual field. Such 
interacts one would probably have a time-dependent cohpnnent as mil at a 
DC component 

IS} Doxiwiaing th B goodnesa of f 1 1 of a solution uou I d have to be 
imp fa Han ted by a local goodness-of-f 1 1 function, that measures the 
proportion of low-level Symbole that ha^e successfully deen Pound to a 
disparity value, and ubicb affects the confidence level of the bindinge 
In that local region., 

Pi ecus* ion 
I Shall not attempt in thifl note to tferiv* anjj properties of the 
above ay a tem. [ have baen unable to Irake much progress u i th an analytical 
approach to tha prpbleti. and the anounf of time required for a 
computational atudy is v B rg large. The a^oacn aet nut hare doss however 
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illustrate [al hou the disparity computation nay be uiM -founded; fbl the 
importance of lou-level symbols. in the formulation; and lc) how the 
important constraints may at least In a fl*nar&l way fc„ represented by 
connexions ui th a s t r-& i yh I f oruard gsoeetry. This kind of geometric*! 
arrangement is one that it ia becoming possible to detect. 

The second large issue concerns tha uay in which disparity 
information may be used. It It one thing to anaign dieparity veluee to 
1 0»i- level symbols, and quite another to divide up an intaga Into regions 
on the basis of disparity information, alone, and conpute a description of 
the spatial ewtent of each region. Far example, in any stereogram, the 
orientation information associated urth the sM 1 1 squares or groups of 
squares lhat are actually Hatched beara no relation lo tha orientation of 
the edge at which disparity changes. One ua u D f computing the higher, 
induced edges would of Course be to treat disparity like Intensity, and 
subject its valuea to a prnceaa lite that used to obtain a row-level 
Symbolic description from an Intenaity array [Mam ]H74b). This Method 
seems somewhat cluesy. however, becauee disparity is not the only kind of 
Information {excluding Intensity! fro* which direct iona and boundaries 
may be computed locally. Texture changes are another s^a^ple, and so are 
more abstract outlines, like the envelope of a sparse tree in winter, or 
the boundary defined by a row of s*al I, eeparated bushes acroee a garden, 
Dn e would like to know whether ell of theea preplnie may be dealt ui th by 
a single method that can describe Configurations of "places' In in image 
- these pi acts being identified by a rather simple Kind of local 
measurement Hade on the relevant tyre of inf creation, There seem* to be 
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clear evidence [and a definite COffpLitat \onai need! for auch a mechaniam. - 
ore of its »ain functions being to sat up an orientation in the image at 
a point, to describe conf iguratiOna of pfacea relative to that 
orientation, and re influence the direction relative to which, local 
shapes in an imayo aris described- It is hnuevar far from clear whether 
□ne such mechanise uould suffice tc service all of the denanda of this 
kind, or whether (he a I ighl lu rtif f er rng tetiputat ional requirementa force 
the existence of S number of separate, but similar machaniamg. The 
question ui 1 1 he raised *l»uhar* [llarr i975P. 

A ].: k 1 ■ i.a-j I 'M:irj c r l 1 -i t ; Uork rejcrferJ ie r e'r uas conducted al 1H*? Artificial 
Intel ligence L ahftra tci- .). * Mo^oachuBat ts Institute of Technology research 
pre or am supported In par ( by the Advanced Re as arch Pro jeer a Agency ot the 
Department of Defense and rrorvtored by the Office of Naval ftaaearch under 
Con tract nuiiber fo8001 4-70 -A -0362-0385, 
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